‹ BackNewspeer review

peer review

Musk calls for peer review among top AI labs before model releases
OpenAI
2026-08-09 23:25:08

OpenAI Researcher’s ‘We Don’t Read Papers’ Remark Reignites Debate Over AI Research Quality

An OpenAI researcher’s remark that “we don’t read papers anymore” has sparked fresh debate over how much trust the AI community should place in top conference publications. The discussion picked up after scrutiny of a surprisingly strong ICLR paper led online commenters to question how some headline results are produced. The broader criticism gained traction alongside a large-scale reproducibility review published in July by SAI, a group co-founded by University of Chicago computer science and data science associate professor Tan Chenhao. SAI examined all 168 oral papers from ICML 2026, a conference that received 23,918 submissions and accepted just 168 for oral presentation, or about 0.7%. According to the report, SAI completed 105 full reproductions and found that only 34 papers reproduced more than 40% of their claims, while just eight cleared the 80% mark. The group also pointed to broken code, missing files, incomplete instructions, mismatched results, and four papers that depended on models no longer available. SAI estimated the median cost of fully rerunning one ICML oral paper at about $8,900, with 17 papers costing more than $100,000 and the most expensive nearing $2.2 million. The episode has sharpened a longstanding tension: elite labs may rely less on papers in day-to-day work, but papers still remain a core gatekeeping tool for students and early-career researchers trying to enter those same institutions.

2220
OpenAI Researcher’s ‘We Don’t Read Papers’ Remark Reignites Debate Over AI Research Quality